A Novel Mixed Precision Distributed TPU GAN for Accelerated Learning Curve

Deep neural networks are gaining importance and popularity in applications services. Due to the enormous number of learnable parameters datasets, training is computationally costly. Parallel distributed computation-based strategies used accelerate this process. Generative Adversarial Networks (GAN) a recent technological achievement deep learning. These generative models expensive because GAN consists two trains on datasets. Typically, trained single server. Conventional learning accelerator designs challenged by unique properties GAN, like computation stages with non-traditional convolution layers. This work addresses issue distributing GANs so that they can train datasets over many TPUs (Tensor Processing Unit). Distributed accelerates process decreases time. In paper, Network accelerated using multi-core TPU data-parallel synchronous model. For adequate acceleration network, data parallel SGD (Stochastic Gradient Descent) model implemented TensorFlow mixed precision, bfloat16, XLA (Accelerated Linear Algebra). The study was conducted MNIST dataset for varying batch sizes from 64 512 30 epochs v3 128 × systolic array. An extensive technique bfloat16 decrease storage cost speed up floating-point computations. curve generator discriminator network obtained. time reduced 79% size TPU.